Multi-Objective Scientific-Workflow Scheduling With Data Movement Awareness in Cloud.

Due to serving several purposes simultaneously, running scientific workflows on dynamic environments such as cloud computing, has become multi-objective scheduling. Among these purposes, Cost and Makespan are probably the most two primitive objectives. Another critical factor in a large-scale scientific workflow is tremendous amount of data during execution. Therefore, this work also includes Data Movement as an additional objective as it has a major impact on network utilization and energy consumption in network equipment in cloud data center. In considering these three objectives, this work proposes a framework for scheduling solutions which combines a new nodes clustering technique in Directed Acyclic Graph (DAG) model known as Multilevel Dependent Node Clustering (MDNC) and the multiobjective optimization, Extreme Nondominated Sorting Genetic Algorithm-III (E-NSGA-III). E-NSGAIII is the recent extension of Nondominated Sorting Genetic Algorithm (NSGA-III). Five well-known scientific workflows, CyberShake, Epigenomics, LIGO, Montage, and SIPHT are selected as testbeds, while the commonly known Hypervolume is chosen as the performance metric. In this work, MDNC is also experimented with both NSGA-III. Comparison among three approaches, E-NAGA-III alone, E-NAGA-III with Peer-to-Peer clustering and E-NAGA-III with MDNC are carried out. The superiority of the proposed framework among them and its limitation are discussed

Measuring data locality ratio in virtual MapReduce cluster using WorkflowSim

The data locality is significant factor which has a direct impact on the performance of MapReduce framework. Several previous works have proposed alternative scheduling algorithms for improving the performance by increasing data locality. Nevertheless, their studies had focused the data locality on physical MapReduce cluster. As more and more deployment of MapReduce cluster have been on virtual environment, a more suitable evaluation of MapReduce cluster may be necessary. This study adopts a simulation based approach. Five scheduling algorithms were used for the simulation. WorkflowSim is extended by inclusion of three implemented modules to assess the new performance measure called `data locality ratio'. Comparison of their results reveals interesting findings. The proposed implementation can be used to assess `data locality ratio' and allows users prior to efficiently select and tune scheduler and system configurations suitable for an environment prior to its actual physical MapReduce deployment